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Abstract 


DARPA  has  recently  undertaken  a  research  project  titled  Real-time  Adversarial 
Intelligence  and  Decision-making  (RAID),  which  provides  in-execution  predictive  analysis  of 
probable  enemy  actions.  A  particular  focus  of  the  program  is  tactical  urban  operations  against 
irregular  combatants  -  an  especially  challenging  and  operationally  relevant  domain.  The  RAID 
program  leverages  novel  approximate  game-theoretic  and  deception-sensitive  algorithms  to 
provide  real-time  enemy  estimates  to  a  tactical  commander.  In  doing  so,  the  RAID  program  is 
addressing  two  critical  technical  challenges:  (a)  adversarial  reasoning:  the  ability  to  continuously 
identify  and  update  predictions  of  likely  enemy  actions;  (b)  deception  reasoning:  the  ability  to 
continuously  detect  likely  deceptions  in  the  available  battlefield  information.  Realistic 
experimentation  and  evaluation  is  driving  the  development  process  using  human-in-the-loop, 
wargames  to  compare  humans  and  the  RAID  system.  This  paper  provides  a  discussion  of  the 
techniques  and  technologies  chosen  to  perform  the  adversarial  and  deception  reasoning.  It  also 
provides  details  about  the  experiments  and  experimentation  environment  that  are  used  to 
demonstrate  and  prove  the  research  goals. 

Introduction 

The  Information  Exploitation  Office  (IXO)  of  the  Defense  Advanced  Research  Projects 
Agency  (DARPA)  has  recently  undertaken  a  research  project  titled  Real-time  Adversarial 
Intelligence  and  Decision-making  (RAID)[1],  to  build  tools  capable  of  in-execution  predictive 
analysis  of  probable  enemy  actions.  A  particular  focus  of  the  program  is  tactical  urban  operations 
against  irregular  combatants  -  an  especially  challenging  and  operationally  relevant  domain. 

The  RAID  program  leverages  novel  approximate  game-theoretic  and  deception-sensitive 
algorithms  to  provide  real-time  enemy  estimates  to  a  tactical  commander.  In  doing  so,  the  RAID 
program  is  addressing  two  critical  technical  challenges:  (a)  adversarial  reasoning:  the  ability  to 
continuously  identify  and  update  predictions  of  likely  enemy  actions;  (b)  deception  reasoning: 
the  ability  to  continuously  detect  likely  deceptions  in  the  available  battlefield  information. 
Although  many  types  of  military  operations  can  greatly  benefit  from  the  capabilities  outlined 
above,  the  RAID  program  is  focusing  on  a  well-circumscribed,  intentionally  narrow  but  still  very 
challenging  domain:  in-execution,  tactical  combat  of  largely  dismounted  infantry  (supported  by 
armor  and  air  platforms)  against  a  guerilla-like  enemy  force  in  urbanized  terrain.  Realistic 
experimentation  and  evaluation  drives  the  development  process  using  the  human-in-the-loop. 
Army  0TB  (OneSAF  Testbed)  wargame  to  compare  humans  and  the  RAID  system.  The 
products  of  the  program  have  potential  for  transition  to  Army  military  intelligence  and  battle 
command  systems. 


Motivation 

In  a  number  of  recent  publications,  US  military  leaders  have  called  for  the  development 
of  techniques  and  tools  to  address  the  twin  challenges  of  adversarial  and  deception  reasoning. 


The  US  Air  Force  community  uses  the  term  predictive  battlespace  awareness  [2,  3]  while 
a  related  term,  predictive  analysis,  is  used  in  the  US  Army  community  [4],  Both  refer  to  future 
techniques  and  technologies  that  would  help  the  commander  and  staff  to  characterize  and  predict 
likely  enemy  courses  of  action,  to  relate  the  history  of  the  enemy’s  performance  to  its  current  and 
future  actions,  and  to  associate  these  predictions  with  opportunities  for  friendly  actions  and 
effects.  Both  communities  have  pointed  out  the  lack  of  technologies,  techniques  and  tools  to 
support  predictive  analysis  and  predictive  battlespace  awareness. 


Recent  years  have  seen  the  emergence  of 
capable  tools  for  generating  friendly  courses  of 
action,  e.g.,  the  Mixed  Initiative  Control  of 
Automateams  (MICA)  [5]  and  Course  of  Action 
Development  and  Evaluation  Tool  (CADET)  [6] 
programs.  Given  the  definition  of  available  assets, 
terrain,  and  enemy  information,  missions  and 
rules  of  engagement,  such  tools  generate  a 
detailed,  optimized  plan  of  actions  (Figure  1), 
allocate  and  task-organize  the  assets,  schedule  the 
actions  with  respect  to  applicable  time 
constraints,  and  estimate  the  outcome  of  the 
operation.  They  even  identify  expected  reactions 
of  the  enemy  as  well  as  suitable  counteractions  of 
the  friendly  forces. 


Figure  1.  Advanced  automated  tools  for  battle 
planning  and  management  are  able  to  produce 
detailed  and  optimized  resource  allocations,  task 
times  and  movement  routes.  However,  they  do  not  yet 
explicitly  reason  about  enemy  counteractions.  [7] 


Overall,  such  tools  have  demonstrated  the  ability  to  perform  on  par  with,  or  better  than, 
the  human  staff;  they  help  produce  complex  planning  products  dramatically  faster  (orders  of 
magnitude  faster)  yet  without  loss  of  quality,  as  compared  to  the  conventional,  manual  process. 
The  battle  plans  produced  by  such  automated  tools  look  to  human  reviewers  rather  sophisticated 
and  insightful.  However,  this  emerging  generation  of  battle  planning  and  management  tools  also 
exhibits  serious  shortcomings,  particularly  in  adversarial  reasoning. 


First,  these  tools  have  no  means  to  take  into  account  the  emotional  and  cognitive  aspects 
of  the  battle.  Real  human  warriors,  at  all  levels  of  responsibility,  have  beliefs,  emotions,  desires, 
biases,  preferences,  etc.,  that  contribute  much  into  their  plans  and  actions.  These  emotional  and 
cognitive  aspects  are  complex,  and  they  change  dynamically  as  the  battle  unfolds.  Today’s  tools 
do  not  reason  on  such  factors.  They  also  do  not  take  into  account  the  inevitable  errors  and 
cognitive  limitations  of  the  humans  in  real-world  warfare. 


Second,  today’s  tools  do  not  explicitly  look  ahead,  wargame  or  game-solve  their  plans 
with  a  view  that  the  enemy  may  also  have  insight  into  friendly  courses  of  action  and  may 
counteract  it.  Generally,  they  plan  backward  from  the  key  events  pre-defmed  in  the  human¬ 
generated  high-level  course  of  action.  Unlike  humans,  such  tools  do  not  attempt  to  invent  (even 
in  a  limited  sense)  the  strategy  of  the  battle.  They  merely  fill  in  the  details  (albeit  important  and 
complicated)  in  the  outline  of  the  adversarial  encounter  envisioned  by  the  human.  While 
acceptable  in  some  applications,  this  shortcoming  may  not  be  in  many  others. 
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Third,  much  of  warfare  is  based  on  deception  and  concealment.  Human  commanders 
explicitly  and  continuously  pay  attention  to  the  possibility  that  the  enemy  would  employ 
deceptive  actions  or  conceal  actual  actions  and  assets  in  order  to  manipulate  the  friendly 
understanding  of  the  current  and  future  events.  Human  commanders  and  staff  planners  also 
develop  and  employ  deception  and 
concealment  in  their  own  plans.  Today’s 
tools  (Figure  2)  do  not  explicitly  reason 
about  such  issues. 


Finally,  the  current  generation  of 
C2  tools  does  not  take  into  consideration 
a  very  important  factor:  the  impact  of 
decision-making  processes  and 
organizations  on  the  enemy  (and 
friendly)  actions.  There  are  complex  and 
influential  dynamics  in  command 
decision-making,  in  communications,  in 
propagation  of  uncertainty,  errors, 
confusion,  trust  and  fears  through  the 
formal  and  informal  networks  of  the 
leadership  of  multiple  units  and 
echelons.  All  of  this  is  outside  the  scope 
of  today’s  tools. 


Figure  2:  Battle  planning  tools  like  CADET  [6]  generate  a 
detailed  battle  plan  while  performing  the  action-reaction- 
counteraction  analysis.  Flowever,  they  rely  on  a  pre-specified 
overall  scheme  of  maneuver,  and  do  not  reason  about  deception. 


It  is  these  shortcomings  in  the  state  of  art  that  the  RAID  program  aims  to  mitigate. 


Adversarial  Reasoning 

RAID  focuses  on  approaches  and  challenges  in  what  may  be  encompassed  by  the  term 
adversarial  reasoning:  computational  solutions  to  problems  of  determining  the  state,  intent  and 
actions  of  one's  adversary,  in  an  environment  where  one  strives  to  effectively  counter  the 
adversary's  actions. 

The  subtopics  within  this  subject  include:  belief  and  intent  recognition,  opponent's 
strategy  prediction,  plan  recognition,  deception  discovery,  deception  planning,  and  strategy 
generation.  From  the  engineering  perspective,  the  applications  of  adversarial  reasoning  cover  a 
broad  range  of  practical  problems:  military  planning  and  command,  military  and  foreign 
intelligence,  anti-terrorism  and  domestic  security,  law  enforcement,  information  security, 
recreational  strategy  games,  simulation  and  training  systems,  applied  robotics. 

Naturally,  adversarial  reasoning  is  particularly  important  to  the  domain  of  military 
operations.  In  military  command  and  control,  the  challenge  of  automating  the  reasoning  about 
intents,  plans  and  actions  of  the  adversary  would  involve  the  development  of  computational 
means  to  reason  about  the  future  enemy  actions  in  a  way  that  combines: 


♦  the  enemy's  intelligent  plans  to  achieve  his  objectives  by  effective  use  of  his  strengths 
and  opportunities; 

♦  the  enemy's  perception  of  friendly  strengths,  weaknesses  and  intents; 

♦  the  enemy's  tactics,  doctrine,  training,  moral,  cultural  and  other  biases  and 
preferences; 

♦  the  impact  of  terrain,  environment  (including  noncombatant  population),  weather, 
time  and  space  available; 

♦  the  influence  of  personnel  attrition,  ammunition  and  other  consumable  supplies, 
logistics,  communications,  sensors  and  other  elements  of  a  military  operation;  and 

♦  the  complex  interplay  and  mutual  dependency  of  friendly  and  enemy  actions, 
reactions  and  counteractions  that  unfold  during  the  execution  of  the  operation. 

Adversarial  reasoning  is  the  process  of  making  inferences  over  the  totality  of  the  above  factors. 

Although  many  of  the  problems  inherent  in  adversarial  reasoning  have  been  traditionally 
seen  as  belonging  to  the  field  of  game  theory,  we  argue  that  practical  adversarial  reasoning  calls 
for  a  broader  range  of  disciplines:  artificial  intelligence  planning,  cognitive  modeling,  control 
theory,  and  machine  learning  in  addition  to  game  theory.  An  effective  approach  to  problems  of 
adversarial  reasoning  must  combine  contributions  from  disciplines  that  unfortunately  rarely 
come  together.  One  of  RAID’s  research  objectives  is  to  explore  important  close  relations 
between  ideas  coming  from  such  diverse  areas. 

Adversarial  reasoning  is  broader  than  the  military  domain,  and  the  benefits  of  RAID’s 
research  results  extends  beyond  C2  applications.  The  applied  communities  (practitioners, 
engineers,  developers)  interested  in  adversarial  reasoning  certainly  include  military  planners  and 
analysts  as  well  as  the  intelligence  community.  Also,  those  who  develop  applications  and 
processes  related  to  anti-terrorism  and  domestic  security  and  law  enforcement  would  share 
similar  interests.  Other,  less  obvious  communities  of  practitioners  include  those  concerned  with 
financial  fraud  detection  and  information  security.  They  would  also  benefit  from  a  better 
understanding  of  what  and  how  the  opponent  thinks  while  preparing  and  executing  financial 
fraud  or  intrusions  into  an  information  system. 


Developmental  and  Experimental  Approach 

The  RAID  system  (Figure  3)  is  composed  of  two  major  components:  the  Adversarial 
Reasoning  Module  and  the  Deception  Reasoning  Module.  The  purpose  of  the  Adversarial 


Reasoning  Module  is  to  generate,  either  on- 
demand  or  in  response  to  battle  situation 
changes,  predictions  of  Red  (enemy)  actions  and 
assumption  about  Blue  (friendly)  actions. 
Continually  observing  the  evolution  of  the 
battlefield  and  the  evolution  of  the  predictions 
made  by  the  Adversarial  Reasoning  Module,  the 
Deception  Reasoning  Module  infers  possible 
concealed  enemy  force  elements  or  movements 
of  elements,  incorrectly  identified  enemy  assets. 


Figure  3:  Key  components  of  the  RAID  program. 


decoys,  and  actions  designed  to  mislead  friendly  forces. 

The  development  of  the  RAID  system  is  driven  by  a  rigorous  schedule  of  increasingly 
difficult  and  realistic  experiments  using  the  0TB  wargaming  simulation  system.  The  purpose  of 
the  experiments  is  to  explore  the  ability  of  RAID  to  make  effective  estimates  of  enemy  actions 
and  assumptions  about  friendly  counteractions  (move-countermove  reasoning),  as  compared  to  a 
human  staff. 

To  focus  the  experimental  development  process,  the  RAID  program  concentrates  on  a 
particularly  demanding  and  operationally-challenging  domain:  tactical  operations  in  an  urban 
environment  against  dismounted  irregular  combatants.  The  complex  urban  terrain  offers  a  high 
density,  as  well  as  fragmentation,  of  threats  and  opportunities  for  forces  [8].  Further,  the  terrain 
itself  is  dynamic  because  it  is  continually  modified  by  human  actions  (barricades  in  the  streets, 
holes  in  the  walls,  etc.). The  presence  of  non-combatants  on  the  battlefield  must  be  explicitly 
considered  and  collateral  damage  minimized.  Fire  and  maneuver  of  forces  are  not  the  only 
actions  that  must  be  carefully  considered.  Intelligence  gathering,  communications,  and  logistics 
(including  casualty  evacuation)  are  tightly  coupled  with  fire  and  maneuver.  The  scale  of  the 
computational  problem  is  immense  and  yet  solutions  must  be  generated  in  near  real-time. 

The  RAID  experiments  aim  to  approximate  the  complexity  of  the  target  environment  to 
the  best  extent  possible.  The  core  of  the  experimentation  testbed,  the  Combat  Simulation  System, 
is  based  on  the  proven  Army  simulation  and  training  system,  0TB.  Certain  modifications  to  the 
existing  system’s  interfaces  and  entity  behaviors  are  being  implemented  to  meet  the  needs  of 
RAID  experimentation. 


In  the  experiments,  the  RAID  system 
performs  the  following  functions:  reads 
Red/Blue  situation  from  the  Combat 
Simulation  System;  accepts  guidance  from 
the  Blue  commander  (priorities,  key 
objectives,  etc.);  on  demand,  estimates  the 
likely  actions  of  Red  and  assumed  actions  of 
Blue  for  the  next  X  minutes  of  wargame 
time;  completes  every  new  estimate  rapidly; 
presents  the  estimate  to  Blue  commander  as 
overlay  graphics. 

A  typical  series  of  experiments  (Figure  4)  consists  of  a  number  of  benchmark  games 
(without  RAID)  and  a  similar  number  of  test  games  (with  RAID).  Control  of  Red  entities  is 
performed  by  a  Red  cell  of  5-7  experienced  human  wargamers.  Control  of  Blue  entities  in  the 
benchmarking  games  is  performed  by  a  Blue  cell  of  5-7  human  wargamers.  Control  of  Blue 
entities  in  the  test  games  is  by  a  Blue  cell  of  only  1-2  human  wargamers  supported  by  RAID. 

In  all  series  of  experiments,  overall  complexity  of  the  problem  is  varied  by  adjusting 
number  and  granularity  of  Red/Blue  units,  and  restrictions  on  the  set  of  available  actions  and 
weapon  types.  It  is  estimated  that  the  complexity  measure  (the  size  of  the  search  space)  of  the 
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Figure  4.  RAID  experiment  setup. 


Phase  I  problem  exeeeds  rapidly  growing  even  further  in  the  following  phases.  For 

comparison,  a  chess  game’s  complexity  is  on  the  order  of  10^^. 


For  the  purposes  of  these  experiments,  success  of  the  Blue  force  is  measured  by  the  rate 
of  progress  toward  the  mission  accomplishment  (e.g.,  advancing  to  or  clearing  the  specified 
objective);  Red  personnel  casualties;  avoidance  of  friendly  losses  and  collateral  casualties. 
Success  of  the  Red  force  is  measured  by  delaying  the  Blue  force  and  causing  Blue  casualties. 
Success  of  the  Blue  force  is  considered  an  effectiveness  metric,  and  scores  are  compared 
between  the  benchmark  and  test  games. 

Comparison  between  the  benchmark  series  of  experiments  and  the  test  series  also  provide 
other  rigorous  quantitative  measures  of  RAID’s  capabilities  compared  to  those  of  human  experts. 
Among  the  experiment  metrics,  accuracy  is  particularly  important:  the  number  of  wrong 
predictions  made  by  RAID,  expressed  as  a  fraction  of  total  predictions  and  compared  statistically 
to  the  same  measure  of  human  expert  performance.  Typical  predictions  refer  to  tangible 
estimates  used  in  the  practice  of  military  intelligence,  such  as  location,  strengths  and  actions  of 
an  enemy  unit  at  a  particular  time  interval  in  the  future.  Wrong  predictions  also  include  false 
positives  -  Red  actions  that  are  predicted  but  do  not  occur  and  false  negatives  -  Red  actions  that 
occur  but  are  not  predicted. 


Key  Technology  Themes 

Three  themes  are  particularly  salient  in  adversarial  reasoning.  Faced  with  an  intelligent 
adversary,  a  decision  maker,  whether  human  or  computational,  often  must  begin  by  using  the 
available  information  in  order  to  identify  the  intent,  the  nature  and  the  probable  plans  of  the 
adversary.  Hence  the  first  key  theme  of  adversarial  reasoning  -  opponent’s  intent  and  plan 
recognition.  Further,  a  capable  adversary  is  likely  to  conceal  his  plans  and  to  introduce  crucial 
deceptions.  Therefore,  the  second  theme  -  deception  discovery  -  focuses  on  detection  of 
concealments  and  deceptions.  Finally,  having  made  progress  in  identifying  an  adversary's  intent 
and  guarding  himself  against  possible  deceptions,  the  decision  maker  has  to  formulate  his  own 
plan  of  actions  that  takes  into  account  potential  counteractions  of  the  adversary  -  and  this  is  the 
third  theme,  strategy  formulation. 

Recent  years  have  seen  a  dramatic  rise  in  the  capabilities  of  techniques  relevant  to 
adversarial  reasoning,  making  potential  solutions  relevant,  for  the  first  time,  to  problems  of  a 
practical  scale  and  complexity.  The  1950's  and  1960's  saw  critical  developments  in  the 
understanding  of  Game  Theory,  a  key  element  of  adversarial  reasoning.  The  game  problems  are 
tremendously  more  complex  than  those  for  systems  without  antagonistic  inputs.  Until  recently, 
game  formulations  of  practical  problems,  with  the  attendant  level  of  detail  and  scale,  resulted  in 
a  degree  of  complexity  that  could  not  be  satisfactorily  handled.  Today,  however,  there  are  claims 
of  computational  techniques  that  offer  the  promise  of  robustness  and  scalability  appropriate  for 
practical  applications.  Furthermore,  there  has  been  a  dramatic  rise  in  the  maturity  of  technical 
approaches  that  address  the  cognitive  aspects  of  adversarial  reasoning,  particularly  the  means  to 
model  how  an  adversary  perceives  a  situation,  reflects  on  what  the  opponent  might  perceive  and 
do,  and  decides  on  a  course  of  action. 


Technology  Theme:  Recognition  of  the  Opponent’s  Intent 

In  formulating  the  predictions,  RAID  takes  into  account  such  factors  as  high-level 
objectives,  the  intents  and  preferences  of  the  friendly  and  enemy  commanders,  physical 
capabilities  and  needs  of  the  assets  available  to  both  sides,  mutual  influence  of  actions  of  Blue 
and  Red  forces,  terrain,  weather,  non-combatants,  cultural  and  doctrinal  aspects,  psychological 
factors  affecting  troops  and  commanders,  prior  evolution  of  the  operation,  etc.  It  is  critical  for 
RAID  to  consider  cognitive  and  emotional  factors.  Real  human  warriors,  at  all  levels  of 
responsibility,  have  beliefs,  emotions,  desires,  biases,  and  preferences,  that  contribute  much  into 
their  plans  and  actions.  They  are  complex,  and  they  change  dynamically  as  the  battle  unfolds. 

One  of  the  technical  approaches 
RAID  is  exploring  for  this  purpose  is  an 
integration  of  cognitive  modeling  tools 
(Figure  5)  with  pheromone-like  modeling 
of  the  combatant’s  perception  of  the  battle’s 
threats  and  opportunities.  This  approach 
uses  a  cognitive  framework  [10]  to  model  a 
fighter’s  cognitive  and  emotional  state 
(beliefs,  desires,  and  intents)  and  includes 
cultural  and  doctrinal  preferences.  This 
cognitive  framework  is  then  augmented 
with  pheromone-based  algorithms  [9], 
using  “ghosts”  to  traverse  multiple 
trajectories  in  the  solution  space  to 
converge  on  a  solution. 


Figure  5.  One  approach  -  a  cognitive  framework  with 
pheromone-based  algorithms  [9]  that  extrapolate  an 
agent's  past  behaviors  and  mental  states  into  the  future. 


Technology  Theme:  Discovery  of  the  Opponent’s  Deceptions 

While  continually  observing  the  evolution  of  the  battlefield  and  the  evolution  of  the 
predictions  regarding  enemy’s  future  actions,  RAID  infers  possible  concealed  enemy  force 
elements  or  movements  of  elements,  incorrectly  identified  enemy  assets,  decoys,  and  actions 
designed  to  mislead  friendly  forces. 

If  available,  RAID  uses  user-provided  estimates  regarding  overall  strength  of  concealed 
enemy  assets,  and  types  of  most  likely  deceptions.  In  formulating  its  estimates  of  enemy 
concealment  and  deception,  RAID  considers  the  state  of  Red  knowledge  about  the  Blue,  the  Red 
beliefs  about  Blue  sensor  capability,  the  known  Red  tactics  of  concealment  and  deception,  the 
costs  and  efforts  of  actions  and  measures  involved  in  execution  of  concealment  and  deception, 
and  the  ability  of  the  Red  to  use  non-combatants  for  the  purposes  of  concealment  and  deception. 

RAID  generates  several  alternative  estimates  of  Red  concealments  and  deception,  if 
multiple  alternatives  are  indeed  likely.  Each  estimate  is  accompanied  by  its  likelihood,  and 


assumptions  on  which  it  is  based.  Elements  of  the  estimates  are  linked  to  the  underlying 
evidenee. 


To  produee  sueh  deeeption  estimates,  RAID  employs  several  teehniques.  One  of  them  is 
based  on  a  risk-sensitive  estimation  [11,  12],  Preliminary  results  indieate  this  approach  is  superior 
to  approaehes  such  as  Bayes/Kalman  filter 
estimators.  The  teehnique  is  supplemented  by 
utilizing  a  non-symmetrie  evaluation  funetion,  whieh 
models  the  goals  and  values  of  the  Blue  and  Red 
team,  as  modified  by  input  from  the  eommander. 

Another  technique  is  a  deeeption  robustness 
estimator  that  also  takes  advantage  of  the  non- 
symmetric  evaluation  funetion.  It  deals  explicitly 
with  the  presence  of  potentially  antagonistic  action 
on  the  part  of  the  opponent,  by  searehing  to  uncover 
deception  aetivities  that  may  be  part  of  a  long  term 
enemy  plan  (Figure  6).  It  recognizes  that  these 
adversarial  aetions  may  be  affeeting  both  the 
dynamies  of  the  unfolding  operation  and  the 
observations  that  are  obtained. 


Figure  6.  A  small  scale  prototype  [12]  uses 
deception  robustness  to  intercept  a  stealthy  Red 


Technology  Theme:  Identification  of  Strategies 

The  key  funetion  of  the  RAID  system  is  to  generate,  either  on-demand  or  in  response  to 
battle  situation  changes,  predietions  of  Red  actions  and  assumption  about  Blue  actions.  As 
information  regarding  battlefield  situation  (locations,  strengths,  postures,  actions,  etc.)  of  enemy 
and  friendly  troops  beeomes  available  or  changes,  either  in  the  deliberate  IPB  and  wargaming 
mode  or  during  the  exeeution  of  the  operation,  RAID  generates  a  new  or  modified  set  of 
predietions,  including  most  dangerous  and  most  likely  prediction,  each  characterized  by  its 
likelihood.  These  detailed  predietion  look  forward  anywhere  from  30  to  300  minutes  (as 
speeified  by  the  user)  from  the  current  moment,  ineluding  sequence  of  actions,  situated  in  time 
and  spaee,  performed  by  the  enemy  foree. 

Beeause  the  aetions  of  Red  and  Blue  forces  are  elosely  connected  and  influenee  each 
other,  RAID  also  necessarily  generates  its  estimates  of  the  friendly  actions  similar  to  the 
predietions  of  enemy  actions.  These  ean  be  seen  as  assumption  or  reeommendations  regarding 
the  friendly  eourse  of  action.  Although  the  primary  function  of  RAID  is  to  anticipate  Red 
actions,  the  capability  to  suggest  Blue  aetions  is  a  natural,  valuable  byproduct  that  can  be 
effeetively  utilized  by  an  integrated  C2  /Intel  system. 

The  set  of  predictions  should  be  broad  enough  to  provide  the  eommander  with  a  sense  of 
possible  alternative  futures,  and  yet  small  enough  that  it  can  be  rapidly  reviewed  in  the  tempo  of 
tactical  combat.  In  particular,  RAID  should  provide  a  suitable  abstraetion  of  eaeh  alternative 
predietion  so  that  it  could  become  a  basis  for  displaying  graphically  as  a  rapidly  comprehensible, 
simplified  sketeh. 


To  produce  such  detailed  predictions  of  Red  and  Blue  (coupled)  actions,  RAID  is 
exploring  several  technologies.  One  of  them  is  Linguistic  Geometry  [13]  where  an  algorithm 
takes  a  Blue  goal  (e.g.,  capture  an  objective,  destroy  a  target)  and  a  Blue  asset,  and  constructs  a 
group  of  multiple  sequences  of  actions  that  can  lead  to  that  goal;  then  it  constructs  a  group  of 
countermoves  that  Red  can  use  to  counteract  the  Blue  action  (the  reaction),  and  then  constructs 


the  Blue  counteractions  (Figure  7)  and  so 


Figure  7.  A  small  example  of  building  action, 
reaction,  counteraction  clusters  in  the 
Linguistic  Geometry  [13]  approach. 


on.  It  is  shown  that  the  construction  of  multiple 
sequences  is  rather  computationally  inexpensive. 
All  such  sequences  of  actions  are  then  organized 
into  clusters.  The  algorithm  then  repeats  this 
process  with  the  remaining  goals  and  assets  and 
generates  more  clusters.  There  is  a  relatively 
small  space  of  such  clusters  and  the  search  for 
the  best  moves  within  this  space  of  clusters  is 
dramatically  less  expensive  than  the  search 
within  the  enormous  space  of  individual  moves 
and  board  positions  used  in  conventional  gaming 
techniques.  Such  approaches  may  produce 
effective  solutions  for  adversarial  reasoning 
problems  of  a  practical  scale  and  complexity  as 
targeted  by  the  RAID  program 


Future  Applications  and  Deployment 


In  the  future,  as  a  military  operation  is  being 
executed  (Figure  8),  the  information  from  the 
battlespace,  such  as  location,  strengths  and  postures  of 
enemy  and  friendly  troops,  is  rapidly  delivered  to  a 
military  intelligence  system.  With  today’s  proliferation 
of  Blue  force  tracking  devices  and  unmanned  air  and 
ground  sensors,  one  envisions  that  the  latency  of  such 
information  would  be  measured  in  minutes.  This  fused 
sensor  data  would  identify  locations  of  some  of  the 
enemy  units  and  some  attributes,  such  as  type,  size,  and 
posture.  It  is  understood,  however,  that  the  fog  of  war 
would  remain  thick  -  in  spite  of  the  proliferation  of 
sensors  and  improved  fusion  techniques.  The 
battlespace  information,  especially  the  enemy 
information,  would  remain  incomplete  and  potentially 
deceptive. 

While  continually  monitoring  the  changes  in  the 
battlespace  state  as  the  information  unfolds,  the  RAID 
system  periodically  or  on  request  generates  predictions 
of  enemy  actions  and  presents  them  in  a  user-friendly. 


Figure  8.  RAID  is  intended  to  provide 
tactical  leaders  with  real-time  predictive 
estimates  of  enemy  actions  - 
continuously  and  in-execution. 


rapidly  comprehensible  format.  These  predictions  are  delivered  to  the  user,  such  as  a  company 
commander,  via  computer  displays  in  combat  vehicles  or  on  personal  digital  assistant  (PDA)-like 
devices  for  dismounted  personnel. 

RAID  output  products  include  several  possible  alternative  enemy  courses  of  action, 
worked  out  in  requisite  detail,  ranked  in  the  order  of  likelihood,  and  presented  as  graphic 
overlays  with  brief  textual  notes  and  with  an  explanation  of  assumptions  about  the  friendly 
course  of  action.  The  scope  and  details  of  the  products  are  tailored  for  each  individual  user,  his 
current  situation  and  area  of  responsibility.  RAID  products  are  designed  to  be  unobtrusive  to  the 
user.  The  user  may  elect  not  to  see  them  at  all,  or  to  see  them  occasionally  on  request;  he  may 
use  them  extensively  or  ignore  them  entirely.  If  time  and  situation  permits,  the  user,  at  his 
discretion,  may  input  to  RAID  additional  information,  such  as  his  updated  intent  and  friendly 
scheme  of  maneuver  for  the  upcoming  phase  of  the  operation  or  his  own  estimates  of  enemy 
intent.  RAID  uses  this  additional  information,  when  available,  to  fine-tune  its  predictions.  In  the 
absence  of  such  input,  RAID  makes  do  with  its  own  assumptions  and  estimates.  In  no  case  does 
RAID  become  an  additional  burden  on  the  user’s  time  and  attention. 

Although  military  intelligence  is  one  area  where  RAID  capabilities  are  particularly 
relevant,  other  applications  of  such  technologies  are  also  attractive.  Thus,  tools  for  friendly  COA 
preparation  and  real-time  battle  management  can  benefit  from  RAID’s  adversarial  perspective. 
Further,  developers  of  military  simulation  and  training  systems,  as  well  as  developers  of 
commercial  entertainment  games,  are  always  striving  for  a  more  realistic  and  intelligent 
opposing  force  within  their  respective  systems.  To  a  significant  extent,  they  can  benefit  from 
advances  in  adversarial  reasoning  offered  by  the  RAID  program.  Finally,  a  less  obvious,  but  very 
relevant,  area  of  practical  applications  is  military  robotics.  In  order  to  survive  and  be  effective  in 
a  hostile  environment,  a  robot  (e.g.,  a  highly  autonomous  unmanned  aerial  vehicle)  should 
reason  about  the  likely  actions  of  its  adversaries. 
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Program  Summary 

Predict  probable  enemy  actions  in  urban  ops 


Problem: 

•  Provide  predictive,  anticipative  analysis  of  enemy 
future  actions 

•  I  dentify  attempts  to  conceal  assets  and  actions  and  to 
deceive 

•  Monitor  and  continuously  confirm,  or  not,  and  update 
the  predictive  analysis 


Solution: 

•  Generate  sets  of  Red  predictions  (including  most 
dangerous  and  most  likely  courses  of  actions)  and 
recommendations  for  Blue  courses  of  action 

•  Identify  probable  enemy  deceptions,  decoys,  feints, 
etc.,  concealed  enemy  assets,  movements  and  actions 
within  the  currently  available  information 


Approach: 

•  Leverage  game-theoretic  and  cognitive  model 
approaches  to  generate  anticipations  and 
counteractions 

•  I  mplement  deception  robust  estimation  techniques  to 
detect  enemy  deceptions 

•  Experimental  proof  of  predictive  capabilities:  human- 
in- the- loop  OneSAF- based  wargames  compare 
humans  and  RAI D 

•  I  ntegrate  the  predictive  analysis  tools  into  warfighter's 
C2  and  intelligence  support  systems 


actions 
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Military  Rationale:  need  for  predictive 
analysis  technology 


Today's  Automation: 

•  Detailed  optimization  of 
allocation,  times,  routes 

•  No  attempt  to  infer  or  to 
influence  the  actions  of 
the  Red 


Humans: 

•  Focus  on  predicting  and 
impacting  actions  of  the 
Red,  by  deception  and 
exploiting  errors 


Army  -  Predictive  Analysis 

#■  "  ..must  provide  ...  running 
estimate ...  incorporating 
predictive  analysis  ..."  —  FCS 
ORD  (1064,  3153,  3465) 

#  "...shall  predict  near-future  enemy 
positions  and  actions  at  intervals 
..."  -  DCGS-A  ORD  (127) 

#■  "...shall  have  tools  ...  for 

performance  of  semi- automated 
predictive  analysis  ..."  —  TRADOC 
Force  Operating  Capabilities  (FOC) 
Pamphlet 

Air  Force  -  Predictive  Awareness 

#  "...  means  to  predict  adversary 
intentions  and  anticipate  adversary 
reactions  ..."  —  Combat  Air  Forces 
CONOPS  for  Predictive  Battlespace 
Awareness 

#  ". .  .visualizing  the  future  of  the 
battle  ...  is  the  'sucking  chest 
wound'  of  a  J  FACC  and  his  staff."  — 
LG  Croker,  USAF  (Ret) 
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Enhanced  Operational  Agility  and 
Survivability  with  Predictive  Analysis 


If  predictive  analysis  could  enable  dramatically 
faster  response  times,  what  would  be  the  impact? 

#  We  explored  the  impact  in  a  simulation  wargame: 

•  Urban  environment 

•  Red  irregulars 

•  Blue  Co  attacks  along  2  AAs 

•  Fire  support  helicopters 

#  Assumption:  real-time  predictive  analysis  will  help 
to  preposition  helicopters  and  reduce  the  time 
required  to  respond  to  ground  troops'  calls  for  fire 

#  Outcome:  Reduced  blue  losses  and  reduced  time 
to  complete  the  mission. 

When  we  know  what  the  enemy  is  going  to  do  or  where  he  is  going  to  be, 
we  can  be: 

#  More  aggressive 

#  More  agile 

#  More  proactive 

#  More  effective 

#  More  survivable 

#  More  assured  of  the  outcome 
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The  CADET  -  An  Exploration  in 
Adversarial  Reasoning 


CADET-  Course  of  Action  Developnnent  and  Evaluation  Tool  -  a 

system  for  semi-automated  planning  of  US  Army  ground  operations 
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Using  a  Course  of  Action  sketch  as  the  input, 
the  CADET  develops  a  detailed  plan  and 
presents  it  as  a  synchronization  matrix 
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The  CADET'S  technical  core  is  an  algorithm 
for  tightly  interleaved,  incremental  planning,  ■ 
routing,  time  estimating,  scheduling,  = 

estimates  of  attrition  and  consumption,  and  | 

adversarial  reaction  estimation.  I 
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RAI D  CONORS  for  tactical  urban  ops 
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5)  Predictions  & 
Retommendalions 


Adversarial 
Reasoning  Module 


CMDR  Guidance 
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The  current  RAI  D  system  comprises  deception 
detection,  enemy  cognitive  estimates,  action- 
reaction  reasoning,  and  OneSAF-based  testbed 
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Cognitive  Modeling 


Strengths 

•  Cognitive,  emotional,  cultural,  doctrinal 
modeling  of  fighters  and  leaders 

•  I  ntegration  with  physical  factors 

•  Non-myopic  behaviors,  look-ahead  for 
cognitive  agents 

•  Proven  capabilities  of  key  components 


Approved  for  Public 


Approach 

•  Explicitly  handles  "human"  aspects  of 
battlefield  behaviors;  cognitive  model 
(Bayesian  belief  net)  propagates  relations 
between  actions,  emotions,  goals,  desires 
and  dispositions 

•  Captures  implicit  cultural  and  doctrinal 
preferences 

•  Connects  observed  behaviors  and 
estimated  mental  state;  projects  mental 
state  into  probable  incipient  goals 

•  Pheromone-analogy  algorithm  prunes  and 
clarifies  past  mental  state  of  the  enemy 
by  fitting  past  behaviors 

•  Projects  future  "broad-brush"  physical 
behavior  and  mental  state  evolution  by 
exploring  multiple  roll-outs  (ghosts) 

•  Approximates  fighters'  look-ahead,  avoids 
being  myopic,  limits  need  for  knowledge 
bases 
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Deception  Reasoning 


•  Rigorous,  novel  theoretical  foundation 

•  Demonstrated  in  two  small-scale  prototypes 

•  Avoids  extensive  knowledge  bases 


Approach 

Deception  robustness  estimator  applies 
stochastic  game  theory  to  state 
estimation  to  discern  underlying 
deception  strategies 

Combines  several  considerations: 
observations,  cost  for  Red  to  deceive, 
value  to  Red  if  deception  works 

Novel  risk-sensitive  theory  for 
recognition  and  analysis  of  deception 
potentials  and  likelihoods 

I  ncludes  limited-cognition  technique  to 
detect  no-concealment  feints  and 
demonstrations 

Non- symmetric  evaluation  function: 
value  functions  produces,  initially 
through  SME  heuristics,  then  through 
automated  learning 
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Action- Reaction  Reasoning 


Strengths 

Substantial  theoretic  basis 

Fully-implemented,  general-purpose  gaming 
engine  worked  in  several  different  domains 

Prototypes  confirm  feasibility  of  large-scale 
real-time  performance 

Avoidance  of  large  knowledge  bases 

Strong  role  of  terrain  and  other  physical  factors 

I  ncludes  elements  of  deception  reasoning 


Approach 

Novel,  highly  efficient  abstraction 
(linguistic  geometry)  of  action  space  for 
non-zero-sum  game  solution 

Small  number  of  general-purpose 
heuristics  guide  low-branching  search 

Multiple  worldviews  reflect  partial 
observability  of  Red  and  Blue 

Cultural  and  SOP  preferences 
accounted  via  features  of  abstracted 
action  space 

Elements  of  Deception  Reasoning  via 
forming  a  solution  for  Red  in  Red's 
partial  worldview 
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Experinnental  Approach:  increasing 
capability  measured  against  human  operators 


Blue  Cell 

4  personnel  w/o  RAID 


Experiment 


Switch 


Blue  Cell 
2  personnel 
w/  RAID 


Control  Cell  (3  personnel) 
enforces  realism  and  integrity  of  the  wargame 


Data  collection  and  analysis  cell  (2  personnel) 
computes  scores  and  predictive  accuracy  w/  and  w/o  RAI D 


Commands 


Situation 


Red  Cell 
4  personnel 

Commands 
agile  and 
aggressive 
Red  Force 


Commands 


Ki 


Situation 


•  This  series  was  9  benchmark  games  (without 
RAID)  and  9  test  games  (with  RAID),  duration  2  hrs 

•  Simulation  software:  0TB 

•  3  mission  types:  point  attack,  zone  attack,  point 
defense 

•  Wargame  scores:  mission  completion;  enemy 
destroyed:  friendly  losses  and  distance. 
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Results  of  the  first  experiments  are 
remarkably  positive 


PAIRED  RUN  METRIC  SCORE 


100 


90 


80 


70 


60 


50 


♦ 

□ 

■ 

A  / 

y/<> 

■ 

A 

\ 

50  60  70  80  90 

NON-RAID 


r  RAID  1 
Inon-RAIDJ 


100 


Type 

Pair  # 

RAID 

Non-RAID| 

A1 x 

1 

71.91 

69.76 

B2 y 

2 

78.34 

75.07 

C1 z 

3 

84.66 

86.81 

A2 y 

4 

74.22 

78.11 

B3 z 

5 

83.04 

73.35 

B4 x 

6 

80.25 

81.41 

C4 y 

7 

94.83 

82.93 

A5  Z 

8 

89.31 

85.92 

B5 y 

9 

81.09 

79.30 

#  of  valid  Run  Pairs  =  9 

Mean  =  2.777 

StDev=  5.210 

Data  Normal  (tested 
using  Shapiro-Wilk) 

Results  Significance 
Parametric:  92.6% 

Results  Significance 
Non-Parametric:  91.3% 


Phase  I ,  Experiment  1 
1-14  April  2005 
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Experimental  Plan: 


bilities 


Phase  1 

Phase  2 

Phase  3  I 

Thrust 

Action-reaction-counteraction 

Concealment  and  deception 

Breadth,  robustness,  transition 

Experiment  Design 

10  benchmark,  10  test  games, 
compare  scores 

Add:  compare  accuracy  of  predictive 
estimates 

In  CPX-like  setup,  integrated  with 

FCBC2,  ASAS-L  /  DCGS-A 

Location  Exp  1 

Exp  2 

System  Integrator  Site  (Orlando  FL) 
System  Integrator  Site  (Orlando  FL) 

Army  BCBL-L  (Ft  Leavenworth  KS) 
Army  BCBL-H  (Ft  Huachuca  AZ) 

System  Integrator  Site  (Orlando  FL) 
JRTC  MOUT  site  (Ft  Polk  LA) 

Terrain 

Digital  Baku  data 

Digital  Jakarta  (JFCOM  data), 
1,800,000  buildings 

Digital  (Exp  1),  Physical  (Exp  2) 

JRTC  MOUT  site  (Ft  Polk  LA) 

OPFOR 

Up  to  20  teams  of  3  fighters  each  w/ 
small  arms,  RPGs 

Up  to  30  teams  of  3-7.  Add  sniper  5 
rifles,  5  HMGs,  5  MANPADS 

200  fighters,  dynamically  formed 
teams.  Add  10  mortars. 

BLUFOR 

Company-sized  force  w/  5  armored 
vehicles 

Add  air  support  (4  helicopters) 

Add  CAS  (10  2-ship  sorties),  joint 
close  support  fires,  air  mobility 

Terrain  Representation 

Buildings  and  floors,  aggregated 
interiors 

Add  breached  openings  in  bldgs; 
basements,  internal  passages 

Add  underground  corridors  of  mobility, 
overpass,  fences,  walls,  urban  clutter 

Intel  Capabilities 

Full  state  known  to  both  sides 

Observations  by  troops 

Add  UGS  and  UAV  sensors 

Organization 

Flat  organization  of  fixed  small  teams 
with  single  command  node 

Company  w/  three  fixed  platoons 

Dynamic  reorganization  and 
reattachment  (10  events) 

Communications 

Implicit  idealized  instant  broadcast 

Comms  and  info  processing  delays 

Differentiated  nets  with  realistic  delays 
and  sporadic  loss 

Casualty  Mgmt 

Implicit  immediate  evacuation 

Treatment,  delayed  evacuation 

Add  explicit  medevac  actions 

Logistics 

Implicit  continuous  resupply 

Run  out  of  ammo,  delays  in  resupply 

Explicit  resupply  actions 

Civilians 

Random  presence  and  reactions 

Civilians  help  red  resupply,  Intel 

Blue  actions  to  manage  civilians 

Concealment,  Deception 

Feint  movements  and  attacks 

Concealment,  stealthy  moves 

Decoys,  civilians  do  diversions 

Timing 

Game  2  hours,  slower  than  real 

Each  game  lasts  2  hrs,  real  time 

Game  lasts  4-6  hrs,  real  time 

Look  ahead  into  future 

At  least  30  min 

At  least  60  min 

At  least  5  hours 

Problem  Complexity 

over  10**8,000 

over  10**20,000 

over  10*50,000 

Solution  speed 

Within  300  sec 

Within  120  sec 

Within  30  sec 

Key  Gate 

RAID-assisted  small  staff  scores  as 
high  as  large  unassisted 

RAID-assisted  small  staff  scores  as 
high  as  large  unassisted 

RAID-assisted  small  staff  scores  as 
high  as  large  unassisted 
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Program  Plan 


Development  Areas; 

■  System:  l  ntegration  and  Experimentation 

■  Technologies:  l)  Adversarial  Reasoning  2)  Deception  Reasoning 

Three  12- month  phases; 


CY04 


Q1 


02 


03 


04 


CY05 


01 


02 


03 


Phase  1 


04 


CY06 


01 


02 


03 


Phase  2 


04 


CY07 


01 


02 


03 


Phase  3 


Theme  of  the  Phase 


Adversarial 
anticipation  and 
counteraction 


Adversarial 
reasoning  about 
concealment  and 
deception 


Integration  and 
Transition 


Core  Technologies 

■  Adversarial  Reasoning 

■  Deception  Reasoning 


Combf  jiodels 


Anticipate  ffi^mnteract 
Feint^ffi  Btacks 


Partial 


Ration 


Concealm^KIff  Deception 


Breadth  4lr^l)ustnesF 


Hum 


ences 


Transition-!^  M  Extensions 


I  ntegration 
and 

Experi  mentation 


Wi 


llilEi  ^ 


m 


w/opi 


[  astern 


#  Transition  RAI D  into  Army  systems 
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RAI D  Program  Summary 


^Operational  Challenge 

#  I  n-execution  predictive  analysis  of 
enemy  probable  actions  in  urban 
operations 

^Program  Objectives 

#  Leverage  novel  approximate  game- 
theoretic,  deception-sensitive,  and 
cognitive  modeling  algorithms  to  provide 
real-time  alternatives  to  tactical 

commander 

^Technical  Challenges 

#  Adversarial  Reasoning:  continuously 
identify  and  update  predictions  of  likely 
enemy  actions 

^  Deception  Reasoning:  continuously 
detect  likely  deceptions  in  the  available 
battlefield  information 

^Realistic  Evaluation 

#  Human- in-the- loop  OneSAF- based 
wargames  compare  humans  and  RAI  D 

^Transition:  Army  dcgs-a 


Real-time 
prediction  of 
enemy 
actions 
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